1. Field
The present invention relates to testing input/output (I/O) settings in a computer-executable application and in particular to a method and apparatus for producing a benchmark application relating to the computer-executable application for testing input/output—I/O—settings of the computer-executable application.
2. Description of the Related Art
There is an ever increasing amount of parallelism in the computational components in successive generations of High Performance Computing (HPC) architectures. I/O hierarchies are becoming more and more complex in order to meet the storage requirements of applications (programs) running on such systems.
Writing applications to efficiently utilize the storage hierarchy benefits from an in-depth knowledge of each layer of the I/O stack and an understanding of how the choice of different I/O programming styles, Application Programming Interface (API) libraries, and file system settings impact application performance.
Two approaches exist for understanding and testing the performance impact of these choices.
The first approach is to run the application with a parameter sweep of the various I/O settings. This gives an accurate picture of how the different settings affect performance. However, it is an impractical and a time-consuming approach as the full application needs to be run for each test.
The second approach is to run a lightweight I/O benchmark suite. These have been developed to run small timing tests that are representative of the I/O operations performed by the application. The problem with such benchmark suites is that they do not match exactly to the operations performed by the application and are generally a very simplified representation of the application, in terms of the level of detail available for testing for each operation. Also, they do not always capture the full set of operations that are involved, for example any communications necessary for gathering data before performing a write operation are often not captured.
I/O characteristics are becoming an increasingly significant bottleneck in the performance of parallel processing in parallel systems and parallel scientific applications. It is therefore important to ensure that an application uses efficient I/O settings to support scalable data movement between disks and distributed memories.
This is a difficult task as scientific applications, for example, have a wide range of I/O requirements depending on their access patterns, the types and sizes of files that are read and written, and the transaction size (the amount of data transferred each time by a process).
A survey of the I/O requirements of High Performance Computing (HPC) applications revealed the following:
Applications are mostly dominated by sequential reads and writes; random access of file data is rare.                Append-only write operations are the main type of I/O operation.        The I/O transaction sizes (the amount of data to be transferred each time by a process) vary widely—generally ranging from several kilobytes to ×10's of Megabytes.        Many applications have adopted one-file-per-process for I/O rather than parallel I/O.        Most applications use their own file-formats rather than portable, self-describing formats, such as HDF5 or NetCDF.        
In addition to the I/O programming choices in the applications, there are a range of hardware and file systems options available which can have an impact on the overall I/O performance of an application.
It is therefore desirable to improve the speed of testing while reducing the processing burden and inaccuracies when carrying out such tests.